How to schedule a Cloud Function to run in the future with Cloud Tasks (to build a Firestore document TTL)

It frequently comes up that it would be very helpful to have a Cloud Function run at a particular point in time in the future. When the Firebase team released scheduled functions, that provided a solution for regularly recurring functions invocations, like a cron job. But that didn’t exactly address the case where only a single function invocation was needed at a very specific time, or on a delay.

Periodic polling doesn’t always cut it

You could certainly use a scheduled function to run frequently, checking some schedule of your creation (perhaps a database) to see if any work is overdue at the moment of invocation. This works for some cases, but has a few downsides:

• You have to build out that scheduler that describes what work needs to be done, and when.
• A scheduled function can only run as frequently as once per minute. This means that it might execute up to one minute later than you’d like, which might be an unacceptable delay.
• You’re paying for useless, repeated function invocations and scheduler queries that might have no work to do.

In particular, here are some situations that don’t work very well with scheduled polling, and could be helped by more granular scheduling of one-off work to be executed on an exact delay:

• Imagine a reservation system that gives someone exactly 15 minutes to complete a reservation. At the end of those 15 minutes, the reservation system needs to terminate the incomplete reservation, and make it available for others.
• Imagine a game that requires the player to complete a level in exactly 2.5 minutes. It would set a flag that rejects updates from the client after that time, in order to enforce the time limit and prevent cheating.
• Or perhaps a live quiz game that requires players to answer a question in 10 seconds before the time is up and advancing to the next question.

For these types of situations, you’re definitely better off with a cheaper and more flexible approach than a scheduled function. The solution I recommend is using Cloud Tasks to schedule the invocation of a “callback” function at the time the followup work needs to be done.

Cloud Tasks can be used as a one-off scheduler

Cloud Tasks is a fully managed service that allows you to manage the execution, dispatch and delivery of a large number of distributed tasks. It’s very versatile, but for the purpose of this post, we can essentially use it as a scheduling mechanism to invoke an Cloud Functions HTTP trigger at a specific time in the future.

With Cloud Tasks, you first create a queue, optionally configure it, then add tasks to the queue using the provided SDK. An individual task can be configured to make an HTTP request with a payload that describes the work to be done. I like to think of the HTTP function as a kind of “future callback” that receives instructions on what to do at the time it was invoked.

Let’s use Cloud Tasks and Cloud Functions to build a Firestore document TTL

One generalized use case that’s a good fit for Cloud Tasks is giving a Firestore document a TTL (“Time To Live”), otherwise known as an expiration date. The idea is to have the document automatically deleted at some point in the future after it was created. Perhaps the document represented something temporary or transient, and needs to be cleaned up at the expiration time.

For the purpose of this blog, imagine implementing a message board that allows people to post messages to it by writing a document into a Firestore collection called “posts”. What we’ll do is allow for the user to arrange for their messages to expire and be removed from the collection after a configurable amount of time.

An important note about Cloud Tasks limitations: you should familiarize yourself with the documentation on quota and limits. For the purpose of this post, know that you won’t be able to schedule a task to execute greater than 30 days in the future.

I’ll assume you’re already familiar with Firestore and Cloud Functions. I’m using the Firebase CLI to deploy functions written in TypeScript. Here’s a diagram of the system I’m about to describe:

Set up Cloud Tasks

Cloud Tasks is a billed Cloud product, so you’ll need to have billing enabled on your project if it’s not already. A breakdown of the pricing for Cloud Tasks is here (IMO, it’s pretty cheap: you get to schedule 1 million tasks for free every month).

First, Enable the Cloud Tasks API in the Cloud console. You won’t be able to do this in the Firebase console, so you’ll have to get a little comfortable with the Cloud console for this one.

Follow the documentation to create a queue. You’ll need to have the gcloud CLI installed and configured for your Firebase project. It basically involves running a single command to create a named queue. I’ll call mine “firestore-ttl”:

$ gcloud tasks queues create firestore-ttl

gcloud might give you a warning, but you can ignore it as long as the output includes a statement that the queue was created.

Write the functions

With Cloud Tasks set up, you can now start using it in your function code. What we need are two functions, one to trigger when a document is created, and another for Cloud Tasks to invoke when it’s time for it to be deleted. Cloud Tasks provides a client SDK for Node. Install it with npm:

$ npm install @google-cloud/tasks

Now, for the function code, I’ll import the usual Firebase modules in addition to the newly added @google-cloud/tasks module. Notice that I’m using a JavaScript require instead of an import for @google-cloud/tasks because it doesn’t yet support import syntax.

import * as functions from 'firebase-functions'
import * as admin from 'firebase-admin'
const { CloudTasksClient } = require('@google-cloud/tasks')
admin.initializeApp()

Next, I’ll define the Firestore onCreate trigger that gets invoked whenever a document in the “posts” collection is created.

export const onCreatePost =
functions.firestore.document('/posts/{id}').onCreate(async snapshot => {
    // Code discussed below!
})

I need to know when the user intends to expire the document, so I’ll support the addition of two fields in the document that specifies the time of expiration. expiresIn is expressed as a number of seconds from the current time. expiresAt is a timestamp field that gives the exact time of expiration. The code that creates the document can use either one. I’ll express these additions as a TypeScript interface:

interface ExpiringDocumentData extends admin.firestore.DocumentData {
    expiresIn?: number
    expiresAt?: admin.firestore.Timestamp
}

Inside the onCreate callback, I’ll pull the document data out, cast it, and perform some checks. What I want is for expirationAtSeconds to be the time of expiration expressed in epoch seconds, no matter which field was used:

const data = snapshot.data()! as ExpiringDocumentData
const { expiresIn, expiresAt } = data
let expirationAtSeconds: number | undefined
if (expiresIn && expiresIn > 0) {
    expirationAtSeconds = Date.now() / 1000 + expiresIn
}
else if (expiresAt) {
    expirationAtSeconds = expiresAt.seconds
}
if (!expirationAtSeconds) {
    // No expiration set on this document
    return
}

Now I need some configuration for the task queue created earlier:

// Get the project ID from the FIREBASE_CONFIG env var
const project = JSON.parse(process.env.FIREBASE_CONFIG!).projectId
const location = 'us-central1'
const queue = 'firestore-ttl'

Then use the Cloud Tasks SDK to give me a fully qualified path to this queue. queuePath is going to be a string that uniquely identifes the task.

const tasksClient = new CloudTasksClient()
const queuePath: string =
    tasksClient.queuePath(project, location, queue)

To fully configure the task, I need to also give it the URL to my callback function and the contents of the payload to deliver.

const url = `https://${location}-${project}.cloudfunctions.net/firestoreTtlCallback`
const docPath = snapshot.ref.path
const payload: ExpirationTaskPayload = { docPath }

You can tell from the URL that the function is called “firestoreTtlCallback”, deployed to the same region and project as the task queue. If you’re using this code, you’ll have to make sure this URL string is correct for your deployment region and project. The actual URL will be available in the Firebase console after you’ve deployed the function.

ExpirationTaskPayload is just another interface that’s helping me stay typesafe when reading and writing its contents. It describes the JSON data structure that will be serialized into the payload:

interface ExpirationTaskPayload {
    docPath: string
}

Now, build up the configuration for the Cloud Task:

const task = {
    httpRequest: {
      httpMethod: 'POST',
      url,
      body: Buffer.from(JSON.stringify(payload)).toString('base64'),
      headers: {
        'Content-Type': 'application/json',
      },
    },
    scheduleTime: {
      seconds: expirationAtSeconds
    }
}

This configuration is saying the following:

• Schedule a task, for later execution, at epoch time expirationAtSeconds.
• At that time, make an HTTP POST request to the given URL.
• The request should have a JSON content body with the contents of the payload object

Note that the encoding to base64 is just required by the Cloud Tasks API. The function will still receive the raw stringified JSON.

Now, all I have to do is enqueue the task in the queue I created earlier:

await tasksClient.createTask({ parent: queuePath, task })

And that’s it for the Firestore trigger. Of course, we still need to write the HTTP callback function to be invoked by Cloud Tasks at the right time:

export const firestoreTtlCallback =
functions.https.onRequest(async (req, res) => {
    const payload = req.body as ExpirationTaskPayload
    try {
        await admin.firestore().doc(payload.docPath).delete()
        res.send(200)
    }
    catch (error) {
        console.error(error)
        res.status(500).send(error)
    }
})

You can see that it pulls out the parsed JSON body, assumes that it contains an ExpirationTaskPayload type structure, and uses the Firebase Admin SDK to delete the document using the path given in the payload.

You can deploy these functions and exercise them by creating a document in the Firebase console, making sure to include either a expiresIn or expiresAt field in the document. Try an expiresIn value of 5 seconds to see very quickly that everything is working. One quick note about security — HTTP functions are deployed with public access by default when using the Firebase CLI, which could be a problem here, so you’ll want to read more about this later on in this post.

But what if I need to cancel the task?

Let’s say we would like to allow the user to also be able to cancel the future expiration of the document. This is also relatively easy. I’ll implement that with the following additions:

1. At the end of the onCreate trigger, write the new task’s queue path into the same document.
2. Use a new Firestore onUpdate trigger to look for the removal of the expiresAt or expiresIn field.
3. If the field was removed, use the queue path in the document to cancel the task using the queue path.

First, the call to create the task must use the returned result:

const [ response ] =
    await tasksClient.createTask({ parent: queuePath, task })

The ID of the task is in the name property of response. I’ll expand the definition of ExpiringDocumentData to allow for this new field called expirationTask:

interface ExpiringDocumentData extends admin.firestore.DocumentData {
    expiresIn?: number
    expiresAt?: admin.firestore.Timestamp
    expirationTask?: string    // added this property
}

And write it back to the document:

const expirationTask = response.name
const update: ExpiringDocumentData = { expirationTask }
await snapshot.ref.update(update)

Now all we need is the onUpdate trigger to look for field removal, delete the task, and also remove the expirationTask field from the document, which is no longer useful:

export const onUpdatePostCancelExpirationTask =
functions.firestore.document('/posts/{id}').onUpdate(async change => {
    const before = change.before.data() as ExpiringDocumentData
    const after = change.after.data() as ExpiringDocumentData
    // Did the document lose its expiration?
    const expirationTask = after.expirationTask
    const removedExpiresAt = before.expiresAt && !after.expiresAt
    const removedExpiresIn = before.expiresIn && !after.expiresIn
    if (expirationTask && (removedExpiresAt || removedExpiresIn)) {
        const tasksClient = new CloudTasksClient()
        await tasksClient.deleteTask({ name: expirationTask })
        await change.after.ref.update({
            expirationTask: admin.firestore.FieldValue.delete()
        })
    }
})

And that’s about all there is to it. The complete code can be found in this gist, and at the bottom of this article.

But what about security?

You might be thinking: “This is great, but now can’t anyone delete any document in my database if they just know the URL and payload format of my callback function?” And the answer to that is definitely yes. By default, the Firebase CLI deploys HTTP functions so that they can be invoked by anyone with an internet connection. But this is a solvable problem.

To lock down access to the callback function, you will need to make some changes to the configuration of both the function and the task so that they use a service account to authenticate that the caller of the function is allowed to invoke it. This involves making some advanced configuration changes that are not visible in the Firebase console. First, configure the task to use a service account for authentication when invoking the function. For the function, go to the Cloud console and change the Cloud Functions Invoker role to remove “allUsers” and replace it with that service account, or “allAuthenticatedUsers”. (Note that “allAuthenticatedUsers” doesn’t refer to Firebase Authentication users. It’s a Google Cloud Platform concept that stands for all service accounts.). It will be similar to what you see in step 3 of this tutorial. Learn more about IAM for Cloud Functions and security configuration for Cloud Tasks.

Also, if you are allowing users to specify expiration times directly from your app, you should strongly consider writing some security rules to place some limits on the values of expiresIn and expiresAt. For starters, these values only make sense if they specify a time in the future. You might also want to restrict the min and max times a document can expire. Lastly, you probably don’t want users to be able to modify the expirationTask field, which would effectively allow the user to prevent cancelation of the task (though allowing them to know the string value isn’t an immediate security problem).

If you don’t want to expose anything to users about the expiration of the document, you will probably want to use a second, private document to contain all this data, and change the code to create it through some other mechanism controlled by your backend. You’ll probably also want to change the payload of the task to contain both documents to delete, including both the user-facing document, and the private document with the expiration.

The code